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Abstract 

The purpose of this paper was to identify existing criteria that may be considered in evaluating journals in the 
scholarship of teaching and learning in agriculture, natural resources, and the life sciences. This can assist 
faculty authors and evaluators of promotion and tenure cases to explain indicators of the quality of the 
publications. The commonly accepted criteria are: peer review; acceptance rate; longevity; open access 
availability; inclusion in indexing/abstracting services; citation analysis; and expert opinion. These data were 
collected for a representative set of journals which indicated that: acceptance rates for the journals varied 
widely; most of the journals existed for at least 10 years; most of the journals did not have an ISI impact factor 
or EigenfactorTM Score; the ERIC database was the predominant indexing resource; and there were no 
published lists of journals in these subjects compiled from expert opinion. 
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Abstract 

The purpose of this paper was to identify existing criteria that may be considered in 
evaluating journals in the scholarship of teaching and learning in agriculture, natural 
resources, and the life sciences. This can assist faculty authors and evaluators of promotion 
and tenure cases to explain indicators of the quality of the publications. The commonly 
accepted criteria are: peer review; acceptance rate; longevity; open access availability; 
inclusion in indexing/abstracting services; citation analysis; and expert opinion. These data 
were collected for a representative set of journals which indicated that: acceptance rates for 
the journals varied widely; most of the journals existed for at least 10 years; most of the 
journals did not have an ISI impact factor or Eigenfactor™ Score; the ERIC database was 
the predominant indexing resource; and there were no published lists of journals in these 
subjects compiled from expert opinion. 

Key Words: Scholarship assessment; evaluation of journals; agriculture; natural 
resources; life sciences; scholarship of teaching and learning 


Introduction 

The purpose of this paper is to identify existing criteria that may be considered in evaluating 
the journals in the scholarship of teaching and learning (SoTL). Interest in SoTL emerged in 
the late twentieth century because of an increasing focus on accountability and a desire to 
increase the prominence of teaching and learning in higher education institutions. It is 
usually considered to be a subset of individual disciplines (Potter, 2008). Since it is not 
practiced by all faculties, those who conduct research in this area may encounter questions 
related to the assessment of this work, particularly for promotion and tenure decisions. The 
limitations of the existing criteria may point to the need for new criteria to be created for 
emerging multidisciplinary areas. The paper provides data related to these criteria for a set 
of journals in the areas of agriculture, natural resources, and life sciences and discusses the 
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implications. 


Literature Review 

The Scholarship of Teaching and Learning 

During the past twenty years, a strong interest in the public policy arena in institutional 
cost-effectiveness and assessment of learning led to increased scrutiny of faculty priorities 
(Edgerton, 2005; Shulman, 2004). The prevailing paradigm considered research as the 
most important of the three primary roles of faculty: teaching, research, and service 
(Becker & Andrews, 2004; Braxton, Luckey, & Helland, 2002; Finnegan & Gamson, 1996; 
Glassick, Huber, Maeroff, & Teaching, 1997; Hutchings, Huber, & Ciccone, 2011; O'Meara, 
2005). The seminal report of the Carnegie Foundation for the Advancement of Teaching, 
Scholarship Reconsidered (Boyer, 1990), changed that model and defined the role of faculty 
as engaging in four forms of scholarship: discovery, integration, application, and teaching. 
Some of the reasons that faculty engage in SoTL are that: 

- It is an obligation of a professional scholar/educator 
■ It is a means of continuous improvement of teaching 

- It addresses the needs of the public for assessment and accountability (Shulman, 
2000 ). 

Definitions of SoTL either focus on the scholarly practice of teaching or on research on 
teaching (Bowden, 2007; Braxton, et al., 2002; Potter, 2008). Theall considers that SoTL is 
an "extension of the three other types of scholarship into the realm of teaching and the 
legitimizing of research on teaching" (Theall, 2003, p. 415) . There is some consensus that 
communication with the scholarly community is a defining characteristic of SoTL (Bowden; 
Braxton, et al.; Hutchings, et al., 2011). Lee S. Shulman explained that scholarly teaching 
becomes the scholarship of teaching when the work of a member of the faculty becomes 
"public, peer-reviewed and critiqued, and exchanged with other members of our 
professional communities so they, in turn, can build on our work" (Shulman, 2000, p. 50). 

Academic promotion and tenure decisions take into consideration the significance of a 
candidate's publications. Writing about education in a particular discipline and writing about 
research in that discipline co-existed, but they usually did not have equal weight. This may, 
in part, be the result of unfamiliarity with teaching-focused journals in a discipline. For 
instance, in the sciences, scientific research and its publishing venues must be reputable for 
tenure success, but research about teaching in the sciences may not be required or even 
well understood. SoTL is an example of an interdisciplinary area that evolved from 
established disciplines (Hagstrom, 2001). So while this research has always been important 
in the discipline of Education, the SoTL movement raised the profile of research about 
teaching in the subject disciplines. It underscored the scholarly rigor needed to be accepted 
as significant by the academy in a broader swath of disciplines. 

Assessment of the Scholarship of Teaching and Learning 

Once SoTL was identified as an aspect of faculty work that is as important as discovery, 
there was a need for a common understanding among academics of how to assess this 
"new" mode of scholarship. A subsequent Carnegie Foundation report began this discussion 
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(Glassick, et al., 1997). However, this remains an ambiguous area (Bowden, 2007). 
Hutchings, Huber, and Ciccone (2011, p. 120) recently stated that "much remains to be 
done to craft guidelines for evaluation, documentation, and peer review that adequately 
recognize the scholarship of teaching and learning." Academic departments typically lack 
standards for evaluating (O'Meara, 2005). 

One criterion in the evaluation of faculty work is their scholarly contributions and the impact 
of their publications. However, this has not always been the case. A glance at the history 
of higher education in the United States shows a significant change in faculty 
responsibilities, and therefore faculty evaluation during the past 150 years. Early American 
colleges and universities, following the model of English universities, were committed to 
teaching, not research (Eliot, 1901). A change in focus began during the industrial change 
in nineteenth century at the time of the passage the Morrill Act and the establishment of 
land-grant ("Morrill Act of 1890. The Agricultural College Act of 1890. An act to apply a 
portion of the proceeds of the public lands to the more complete endowment and support of 
the colleges for the benefit of agriculture and mechanic arts...", 1890; "Morrill Act. An act 
denating public lands to the several states and territories which may provide colleges for the 
benefit of agriculture and the mechanic arts," 1862). Universities included the goal of 
service to expand and communicate knowledge for general societal improvement, especially 
in the fields of business, agriculture and technology. 

Another shift began in the twentieth century, inspired by the German university tradition of 
discovery of knowledge. After World War II, research as the model for faculty work became 
the major focus. Universities rewarded research by promoting faculty, granting tenure, and 
increasing salaries. During the post-World War II era, especially the 1960's, the major 
concern of universities was to recruit and retain faculty members. By the 1980's, the 
emphasis on research was universal; John Centra reported in his 1977 survey of over 450 
department heads that large research universities emphasized research, although teaching 
was a close second in importance (Centra, 1977). The 1987-88 National Survey of 
Postsecondary Faculty Teaching, Learning and Assessment reported "even schools 
traditionally structured for teaching—liberal arts and comprehensive institutions—now 
followed the research model" (Fairweather, 1993, p. 11). Promotion and tenure reviewers 
began to critically evaluate faculty member's performance. The quantity and quality of a 
faculty member's publications became critical (Centra, 1977). 

The debate of the relative importance of teaching vs. research continued. One of the major 
proponents of bringing the focus back to teaching was Ernest Boyer (Boyer, 1990), past 
President of The Carnegie Foundation for the Advancement of Teaching. He advocated for 
expanding the definition of scholarship to include the application of knowledge. He believed 
that faculty members should share their knowledge through teaching. As mentioned earlier, 
he outlined four separate, but overlapping functions: the scholarship of discovery, the 
scholarship of integration, the scholarship of application and the scholarship of teaching. 

The need to evaluate the scholarship and publications of a faculty member necessitates a 
fair methodology. Evaluation of the journals in which they publish is part of this. 

This leads to the question that is the focus of this article. How can a faculty member or 
academic unit in an emerging field, such as SoTL, determine the quality, impact, or prestige 
of the journals in the field? This may seem like a straightforward determination, but there 
are actually a number of nuanced factors to take into consideration to place a journal into 
context. 
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Criteria for Journal Quality Criteria 

Common methods for ascertaining the quality of a journal are: 

■ Peer review 

- Acceptance rate 

- Longevity 

■ Open access availability 

■ Inclusion in indexing/abstracting services 

- Citation analysis, such as ISI journal impact factor, /7-factor, and Eigenfactor™ Score 

■ Expert opinion or inclusion on a "core list" of journals compiled by experts 

Peer review. One widely accepted indicator of journal quality is peer review (Long, 2010). 
Peer review means that the article has been reviewed by other scholars in the field prior to 
publication. Usually this process is anonymous, or "double-blind;" neither the author nor 
the reviewer is known. Commonly an editor will ask two or three other experts to review 
the article and comment on its acceptability. However, since all research journals use 
some form of this method, it cannot be the sole method of evaluating the relative quality of 
a journal. 

Acceptance Rate. The acceptance rate is the percentage of articles submitted to a journal 
that are accepted for publication. In 2009 Haensly, Hodges and Davenport (2009) studied 
acceptance rates in relation to journal quality in the field of economics and finance. They 
found a relationship between lower acceptance rates and higher citation counts, impact 
factors, and survey-based rankings. This suggests that acceptance rates may be an 
indicator of journal quality. (Haensly, et al., 2009) Fields that have less consensus about 
theories and methods have lower journal acceptance rates than fields that have high 
consensus (Hargens, 1988). However, there are also caveats about considering this factor. 
It is important to understand the context of a journal's publishing pattern and know about 
its editorial policies. Journals that are older or supported by associations in the field may 
receive more manuscripts and therefore have lower acceptance rates. In some cases 
editors may work more closely with authors to revise articles and therefore have higher 
acceptance rates. Journals that have a narrower subject focus have more consensus on 
theories and methods and therefore have higher acceptance rates. 

Longevity. A journal's longevity (length of time it had been published) is a de facto 
indicator of value. Interest in SoTL may have spawned new journals that do not have the 
advantage of longevity that influences the reputation and prestige of journals. 


Open access availability. Open access (OA) journals are those freely available on the Web. 
They have varying business models, including subsidies through author charges or 
sponsorship by institutions. Journals that are not open access usually regain their costs 
through subscriptions. Only those who have personal subscriptions, access to libraries with 
subscriptions, or choose to pay per article can access the articles. As early as 2005 there 
was some evidence that OA journals should be weighted more in journal ranking lists than 
non-open access journals (Ladwig & Sommese, 2005). Eysenbach (2006) did one of the 
first longitudinal studies of OA and non-OA articles. During the first four to sixteen months 
after publication, he concluded that "open access articles are cited earlier and are, on 
average, cited more often than non-OA articles" (Eysenbach, p. 0696). Davis, et al. 
examined eleven scientific journals and found that OA articles "may reach more readers 
than subscription access publishing" (Davis, Lewenstein, Simon, Booth, & Connolly, p. 343). 
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However, they did not find evidence of increased citations during the first year. In a study 
of four disciplines by Norris, et al. (2008), there was a clear citation advantage for OA 
articles, however this advantage varied by discipline. There is some evidence that OA 
journals should be weighted more in journal ranking lists than non-open access journals 
(Ladwig & Sommese, 2005). In a recent meta-analysis of OA citation studies by Swan, 27 of 
31 studies showed a greater rate of citation for open access articles (Swan, 2010). Of even 
greater interest to this study, this analysis examined the higher citation rate by broad 
disciplinary area. Agricultural studies showed an increase of 200-600% in citations with OA, 
greater than any of the other nine areas. 

Inclusion in indexing/abstracting services. Inclusion in major indexing/abstracting services 
such as BIOSIS, Agricola, or Web of Science involves a selection process that can be 
stringent (Paynter, Jackson, & Mullen, 2010). Editorial boards identify the major journals to 
be indexed. One limitation of this method is that a journal must be in existence for several 
years to be considered. 

Citation Analysis. After peer review, the most widely accepted indicator of journal quality is 
the ISI journal impact factor, an outgrowth of citation indexing. Through citation indexes, a 
researcher can find new publications on a topic by identifying articles that cited known 
articles from the past. Citation indexing has a long history dating back to its use in legal 
research in the 19 th century. Modern use of citations to identify key articles originated with 
Eugene Garfield (2006). He published the first edition of Science Citation Index®, (SCI) in 
1961. The purpose of SCI was to identify newer articles, i.e., as an indexing tool. 

The use of citation data to determine journal rankings or "impact" was a direct outgrowth of 
Garfield's citation indexes. Although Garfield first mentioned journal "impact factor" in 1955 
(Garfield, 2006). It was not until 1975 that he and Sher re-sorted the author index of SCI 
by journal name to create the first ranking of journals by citations. This led to the 
publication of Journal Citation Reports (JCR ). Many studies used the ISI journal impact 
factor to identify core collections of journals for specific disciplines (Blessinger & Frasier, 
2007; Blessinger & Hrycaj, 2010; deVries, Kelly, & Storm, 2010; Weissinger, 2010). 

The impact factor has been criticized extensively in the literature, especially when used as 
an evaluation measure for promotion and tenure. Questions centered on its validity, its 
variation among disciplines (Leydesdorff, 2008; Van Nierop, 2009), and a low correlation 
with expert opinion surveys (Serenko & Dohan, 2011). It includes self-citations, and a 
single, highly cited article can strongly influence it (McGarty, 2000). Many articles that are 
very influential are published in journals with lower ISI journal impact factors. Journal 
Citation Reports is selective in indexing journals, especially in the social sciences and the 
humanities; of the 12,000 journals indexed only approximately 2,500 are in the social 
sciences (Social Sciences Citation Index). Altmann and Gorman (1998) concluded that 
impact factor is not reliable as a measure of a journal's importance since impact factors 
vary each year. 

Criticism of the ISI journal impact factor led to the proposal of other metrics. The two most 
common are the Eigenfactor™ Score (hereafter referred to as "Eigenfactor") and the h- 
index. Each has advantages and disadvantages; each attempts to solve problems of other 
metrics. 

The journal impact factor is based on calculating the average number of times the articles in 
a journal have been cited by newer articles. It can be further refined by limiting the range 
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of years during which the citations are counted or the years during which the articles were 
published, by eliminating self-citations, and other variations. ISI's definition of the journal 
impact factor limits the calculation by dividing the number of citations in the census year by 
the number of articles published in the previous two years. An ISI journal impact factor of 
1.0 means that, on average, the articles published one or two year ago have been cited one 
time (ISI's Journal Citation Reports help page). 

The Eigenfactor ranks the influence of a journal rather than an article. It calculates the 
number of times that articles published in a journal during a census period provide citations 
to papers published during an earlier period. Journals generating higher impact to the field 
have larger Eigenfactor scores. The Eigenfactor approach is thought to be more robust than 
the impact factor by considering the significance of those citations. 

A third metric is the /7-index and was developed by physicist Jorge Hirsch (2005). He 
suggested that "a scientist has index A? if A? of his/her N p papers have at least h citations 
each, and the other (N p - h) papers have no more than A citations each" (Hirsch, p. 

16569). The calculation can be applied to journals as well as to authors. 

Citations per article are reported in every record in the Web of Science, Google Scholar and 
Scopus. The ISI journal Impact Factor, the Eigenfactor and the A-index can be retrieved 
through online database searches, although not all through the same database. These 
calculations will vary depending upon the set of journals used in the calculation. For 
example the A-index can be calculated based on journals indexed in the Web of Science or 
those indexed in Google Scholar. 

The Eigenfactor is reported in Journal Citation Reports (JCR) and also at the Eigenfactor 
website and is based on the journals indexed in the Web of Science. 

The A-index is available from the Web of Science and from Harzing's Publish or Perish. 

Each of these sources used a different set of data to calculate the A-index. To obtain the A- 
index as calculated in the Web of Science, search for a journal name in the Web of Science 
in either Science Citation Index or Social Science Citation Index and then click on the 
"Create a report" icon. In these databases the "A-index factor is based on the depth of your 
product subscription and your selected timespan. If your subscription depth is 10 years, 
then the A-index value is based on this depth even though a particular author may have 
published articles more than 10 years ago."(Web of Science Help). Harzing's Publish or 
Perish program uses the citations per article in Google Scholar to calculate the A-index. To 
get the A-index from Harzing's Publish or Perish download the software from their website 
(Publish or Perish). Then use the "journal impact" tab and enter a journal name. The 
calculations are compiled and appear in the results window. 

To summarize this, the journal ISI impact factor and the Eigenfactor based upon the data in 
the Web of Science database, can be obtained from Journal Citation Reports. The A-index, 
based upon Web of Science data, can be obtained from the Web of Science. The citations 
per paper average, which is comparable to the ISI journal impact factor, and the A-index 
based upon the statistics in Google Scholar can be calculated using Harzing's Publish or 
Perish program. 

Expert opinion and core lists of journals. Many academic departments in universities 
develop ranked lists of journals by polling their own faculty (Paynter, et al., 2010). 
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Scholarly associations may compile core lists of journals by surveying their members' 
opinions on journals. In some fields there are few such compilations while in other fields, 
there are many such lists. Surveying experts in a field is a method used for compiling core 
lists of journals (Blake, 1996; Bray & Major, 2011; Goodyear et al., 2009; Kohl & Davis, 
1985; Lamp, 2009; Nisonger & Davis, 2005; Smith & Middleton, 2009; Stankus, Clavin, & 
Joslin, 1999). 

This concept has limitations for an area of discipline-focused SoTL journals, which are only 
loosely affiliated given subject matter. Because of the multi-disciplinary focus of these 
journals, as well as the broad potential audience (e.g., instructors in any of the life 
sciences), a core list has not been compiled and may be impossible to create. 

Combining expert opinion and citation analysis methods. It is not uncommon to use 
multiple methods to compile a core list. The impact factor can supplement the results of an 
expert opinion survey (ABS Launches Academic Journal Quality Guide Version 4; Towns & 
Kraft, 2012; Ugaz, Boyd, Croft, Carrigan, & Anderson, 2010; Youngen, 2011). The 
Australian government took on an ambitious project to assess scholarly journals. This was 
part of the program, Excellence in Research for Australia, initiated by the Australian 
Research Council. The initial list included nearly 20,000 journals allocated among 181 fields 
(Lamp, 2009). The peer reviewed journals in education were evaluated by three criteria: 
esteem percentage, a prestige evaluation developed from responses of over 800 scholars to 
a survey; the ISI journal impact factor; and whether the journal had an international 
editorial board. 


Methodology 

The methodology for this study consisted of these stages: 

■ The identification of relevant journals 

* The identification of evaluative criteria for the journals 

■ Data-gathering about the relevant journals 

Identification of Relevant Journals 

The authors began the process of identifying journals for this study by compiling a master 
list of all SoTL journals included on a list from POD: Professional and Organizational 
Development Network in Higher Education 

( http://www.podnetwork.org/resources/periodicals.htm ). the Committee on Institutional 
Cooperation (CIC), and Ulrichsweb™, a standard resource for information about journals. 
Compilers of the former two lists did not post their selection criteria for inclusion on the 
lists. Ulrichsweb ™ strives to be a comprehensive resource on journals. POD supports 
centers or departments in colleges or universities whose focus is faculty development. 
These centers are resources that faculty instructors can use to develop teaching 
approaches. Many of these centers post lists of journals related to SoTL on their web sites. 
The other lists used for compiling the master list of journals were from land-grant 
institutions that were peer institutions of Purdue University. These were members of the 
Committee on Institutional Cooperation ( http://www.cic.net/Home.aspx) (CIC), a 
consortium of twelve research universities at the time the list was compiled that included: 

■ University of Chicago 

* University of Illinois* 
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- Indiana University 

- University of Iowa 

* University of Michigan 

■ Michigan State University* 

- University of Minnesota* 

- Northwestern University 

■ Ohio State University* 

- Pennsylvania State University* 

* Purdue University* 

- University of Wisconsin-Madison* 

The listed institutions with an asterisk following their names are CIC members that are land- 
grant institutions (http://www.csrees.usda.gov/qlinks/partners/state_partners.html). Land 
grant institutions are public institutions that also conduct research and respond to the needs 
of their home states in areas such as agriculture (Toutkoushian, 2001). Most of the 
universities posted a list of SoTL journals on their web sites. 

The journal titles included on all of these lists and indications of which institutions listed 
them were recorded in a Microsoft Excel spreadsheet. Since the total number of titles was 
large, the authors decided to limit the subject areas of the journals included for this paper 
to agriculture, natural resources, and the life sciences (excluding medicine). The authors 
determined each journal's focus by using the "aims and scope" (or similarly titled) area on 
the journal's website to ensure that articles included and audience addressed met the 
criteria for selection. The authors did not include titles in non-English languages or that 
focused solely on K-12 education. The resulting list contained 36 journals. These are listed 
with their ISSN's in Appendix A. The ISSN is the standardized international code which 
allows the identification of any serial publication, including electronic serials, independently 
of its country of publication, of its language or alphabet, of its frequency, medium, etc. 

Evaluative Criteria for the Journals 

The commonly accepted criteria for evaluating journal quality and impact discussed above 
were applied to the journals selected for this study: 

* Peer-review 

■ Acceptance rate 

- Longevity 

* Open access availability 

■ Citation analysis (including ISI journal impact factor, Eigenfactor, and b-index based 
on Google Scholar data) 

- Inclusion in indexing/abstracting services 

There were no published lists developed by soliciting the opinion of experts in the 
scholarship of teaching and learning in agriculture, natural resources, or the life sciences 
about which journals are the best in the field. Similarly, there were no published lists of 
core journals for those fields. 

Table 1 lists the sources for the data collected. The primary, or original, sources for the 
data are in boldface. The URL's for the resources listed in the table do not provide access to 
the source if a subscription is required. Those affiliated with institutions that subscribe to 
the resources generally may gain access through their institutional library web site. 
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Acceptance rate. The authors requested this information from journal editors by email and 
sent a second email to non-respondents. 

Longevity. The starting date for each journal was initially obtained from Ulrichsweb™ and 
verified against the journal's website. In several instances, the website reflected more 
current information, such as a name change for a journal. 

Open access availability. The open access availability of each journal was determined from 
the publisher's web site and from Ulrichsweb™. 

Inclusion in indexing/abstracting services. Greider (2002) listed the most critical indexes 
for the agricultural sciences. Since several of these were out of scope for this research 
(e.g., indexing government documents, grants, or dissertations), only four remained: 
Agricola, CAB Abstracts, BIOSIS/Biological Abstracts, and Web of Science. ERIC is the 
primary index in the field of Education (Weiner, 2009). 

These were the sources used to determine the current inclusion of the selected journals in 
primary indexing and abstracting sources for agriculture, education, natural resources, and 
the life sciences: 

- BIOSIS Journal Search Subject Categories (BIOSIS PREVIEWS - SUBJECT 
CATEGORIES) 

* CAB Abstracts Serials Cited (CAB Abstracts Serials cited) 

* Web of Science Master Journal List (Master Journal List) 

■ Journals Indexed in ERIC (Journals Indexed in ERIC) 

Indexing information was not available for Agricola because the National Agricultural Library 
is reengineering its production. The number of journals that will be indexed will increase 
greatly and its current status would not be an accurate or useful snapshot. 


Citation analysis. ISI impact factor and the Eigenfactor were identified through Journal 
Citation Reports 2010. The b-index was obtained from a search of Google Scholar on Nov 
8, 2011 using the Publish or Perish (PoP) software program. The search was limited to the 
2008-2011 publication years. 


Table 1 . Sources for Data on Evaluation Metrics for Journals in the Scholarship of Teaching and 
Learning in Agriculture, Natural Resources, and the Life Sciences. 


Criterion 

Sources 



Peer Review 

Journal's web page 

Ulrichsweb™ (http://ulrichsweb.serialssolutions.com/) 

Acceptance Rate 

Contact journal editor; Journal's web page 

Longevity 

Journal's web page 

Open Access 
Availability 

Publisher's web site; Ulrichsweb™ 

(httD://ulrichsweb. serialssolutions.com/): 

Directory of Open Access Journals (httD://www.doai.orq/) 

Impact Factor 

Journal Citation Reports (in Web of Science) 

(httD://thomsonreuters.com/Droducts services/science/science Droducts/a 
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-z/iournal citation reports/! 


Journal's web page 

Google Scholar data 

Google Scholar Citations 

(http ://scholar. google.com/citations?view_op=new_profile&hl=en) 

Publish or Perish database (http://www.harzinq.com/pop.htm) 

Scopus 

http://www.scopus.com/home.url 

Eiqenfactor™ Score 

http://www.eiqenfactor.orq; Journal's web paqe 

rt-Index 

Journal Citation Reports (in Web of Knowledge) 

(■http://thomsonreuters.com/products services/science/science products/a 


-z/iournal citation reports/) 


Journal's web page 

Publish or Perish database (http://www.harzinq.com/pop.htm) 

Scopus (httD://www.scoDus.com/home.url) 

Inclusion in 

Indexing/Abstracting 
Services 

Biological Abstracts Journal List 

(http://science.thomsonreuters.com/cqi-bin/irnlst/ilresults.cqi?PC=BA) 

CAB Abstracts Serials Cited 

(http://www.cabi.org/default.aspx?page=1016&site=170&Did=125&xslttab=2&newtitlesonlv 

=0&letter=*) 

Journals Indexed in ERIC 

(http://www.eric.ed.qov/ERICWebPortal/iournalList/iournalList.isp) 

Science Citation Index Expanded Journal List 

(http://science.thomsonreuters.com/cqi-bin/irnlst/iloptions.cqi?PC=D) 

Expert opinion or 
inclusion on a "core 
list" of journals 

Literature search 

Scholarly associations 


Results 

Table 2 displays the acceptance rate, when available, journal start date (longevity), and 
open access availability for the journals in this study. 

Acceptance Rate 

The journal editors reported this information in a variety of ways, including an average 
acceptance rate for one or more years or a range of averages. Eight journal editors did not 
respond. Two declined to provide an acceptance rate. 

The acceptance rates varied from 7% to 80%. The mean response rate was 41% and the 
median response rate was 44%. This was calculated by substituting 29% for the Electronic 
Journal of Science Education, whose acceptance rate was <30%; 24% for Science Educator, 
whose acceptance rate was <25%; 11% for the Journal of Science Teacher Education, the 
average of its rate that ranged from 7-15%; and 67.5% for the NACTA Journal, the average 
of its rate that ranged from 63-72%. 

Longevity 

All but three (n = 33, 92%) of the journals existed for at least ten years. The newer journals 
began publication in 2002 or 2003. 

Open Access Availability 

Most (n = 26, 72%) of the journals were not available through open access. Eight (22%) 
were completely OA and two (6%) became OA after an embargo period of six months 
( Bioscene: Journal of College Biology Teaching) and one year ( Journal of Agricultural 
Education). 
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Table 2. Characteristics of Selected Agriculture, Natural Resources, Life Sciences Journals Publishing 
SoTL. 


Journal Title 

Acceptance 

Rate 

Longevity 

Open 

Access 

Advances in Physiology Education 

70% A 

1989- 

N 

American Biology Teacher 

35% 

1938- 

N 

Applied Engineering in Agriculture 

76% 

1985- 

N 

Biochemistry & Molecular Biology Education 

60% 

1972- 

N 

Bioscene: Journal of College Biology Teaching 

30% 

1975- 

y* 

BioScience 

45% 

1951- 

N 

Bioscience Education 

no response 

2003- 

Y 

CBE Life Sciences Education 

52% 

2002- 

Y 

Electronic Journal of Science Education 

<30% 

1996- 

Y 

Environmental Education Research 

/\ /\ 

1995- 

N 

Instructional Science 

43% AAA 

1971- 

N 

International Journal of Science Education 

no response 

1979- 

N 

Journal of Agricultural Education 

36% 

1960- 

y** 

Journal of Agricultural Education & Extension 

20% 

1994- 

N 

Journal of Biological Education 

/\ /\ /\ /\ 

1966- 

N 

Journal of College Science Teaching 

35% 

1971- 

N 

Journal of Environmental Education 

15% 

1969- 

N 

Journal of International Agricultural & Extension 

Education 

1Q%AAAAA 

1994- 

N 

Journal of Microbiology & Biology Education 

no response 

2000- 

N 

Journal of Natural Resources & Life Sciences Education 

60% 

1972- 

N 

Journal of Research in Science Teaching 

no response 

1963- 

N 

Journal of Science Education & Technology 

57% 

1992- 

N 

Journal of Science Teacher Education 

7-15% 

1989- 

N 

Journal of sTEm Teacher Education 

60% 

1994- 

Y 

Journal of Technology Education 

44% 

1989- 

Y 

Journal of Technology Studies 

46% 

1974- 

Y 

Journal of Undergraduate Neuroscience Education 

no response 

2002- 

Y 

Journal of Women and Minorities in Science and 

Engineering 

no response 

1994- 

N 

NACTA (North American Colleges and Teachers of 
Agriculture) Journal 

63-72% 

1957- 

Y 

Research in Science & Technological Education 

45% 

1983- 

N 

Research in Science Education 

25% 

1971- 

N 
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School Science & Mathematics 

no response 

1901- 

N 

Science Education 

14.9% 

1916- 

N 

Science Educator 

<25% 

1992- 

N 

Science Teacher Education 

80% 

1991- 

N 

Studies in Science Education 

no response 

1974- 

N 


* After 6 month embargo. 

** After 12 month embargo 
A In 2010; 58% in 2009; 53% in 2008. 

/ ' l/v Editor indicated that acceptance rate can give a misleading impression of quality and so would not provide it. 
In 2010; 27% in 2009. 

/\/\/\/\ Editor declined to provide because final decisions on publication of submitted papers relate entirely to the 
quality of the article (rather than any pre-determined acceptance quota). 

In 2009; 20% in 2008; 18% in 2007. 


Inclusion in Indexing/Abstracting Services 

Table 3 shows the inclusion of the selected journals in primary indexing sources for 
agriculture, education, natural resources, and the life sciences. ERIC included the most 
journals (n = 28, 78%). CAB Abstracts included 6 (17%); Web of Science included 5 (14%); 
and BIOSIS included 2 (6%) of the journals. Five (14%) of the journals were not included 
in any of these indexing sources. 


Table 3. Primary Indexing Sources for Selected Agriculture, Natural Resources, and Life Sciences 
Journals Publishing SoTL. 


Journal 

BIOSIS 

CAB 

ERIC 

Web 

of 

Sci. 

Advances in Physiology Education 

N 

N 

Y 

Y 

American Biology Teacher 

N 

N 

Y 

Y 

Applied Engineering in Agriculture 

N 

Y 

N 

Y 

Biochemistry & Molecular Biology Education 

N 

N 

Y 

N 

Bioscene: Journal of College Biology Teaching 

N 

N 

Y 

N 

Bioscience 

N 

Y 

Y 

Y 

Bioscience Education 

N 

N 

Y 

N 

CBE Life Sciences Education 

N 

N 

Y 

Y 

Electronic Journal of Science Education 

N 

N 

N 

N 

Environmental Education Research 

N 

N 

Y 

N 

Instructional Science 

N 

N 

Y 

N 

International Journal of Science Education 

N 

N 

Y 

N 
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Journal of Agricultural Education 

N 

N 

Y 

N 

Journal of Agricultural Education & Extension 

N 

Y 

Y 

N 

Journal of Biological Education 

Y 

N 

Y 

N 

Journal of College Science Teaching 

N 

N 

Y 

N 

Journal of Environmental Education 

N 

N 

Y 

N 

Journal of International Agricultural & Extension 
Education 

N 

Y 

N 

N 

Journal of Microbiology & Biology Education 

N 

N 

N 

N 

Journal of Natural Resources & Life Sciences 

Education 

Y 

Y 

Y 

N 

Journal of Research in Science Teaching 

N 

N 

Y 

N 

Journal of Science Education & Technology 

N 

N 

Y 

N 

Journal of Science Teacher Education 

N 

N 

Y 

N 

Journal of sTEm Teacher Education 

N 

N 

Y 

N 

Journal of Technology Education 

N 

N 

Y 

N 

Journal of Technology Studies 

N 

N 

Y 

N 

Journal of Undergraduate Neuroscience Education 

N 

N 

N 

N 

Journal of Women and Minorities in Science and 
Engineering 

N 

N 

N 

N 

NACTA Journal (North American Colleges and 

Teachers of Agriculture) 

N 

Y 

N 

N 

Research in Science & Technological Education 

N 

N 

Y 

N 

Research in Science Education 

N 

N 

Y 

N 

School Science & Mathematics 

N 

N 

Y 

N 

Science Education 

N 

N 

Y 

N 

Science Educator 

N 

N 

Y 

N 

Science Teacher Education 

N 

N 

N 

N 

Studies in Science Education 

N 

N 

Y 

N 


Citation Analysis 

Table 4 shows the data related to citation analysis for the journals. Most of the journals 
(n = 22, 61%) did not have an ISI journal impact factor. Fourteen (39%) journals had an 
ISI journal impact factor, which ranged from .09 to 5.51. The mean ISI journal impact 
factor was 1.34 and the median was 0.96. 

Most of the journals (n=25, 69%) did not have an Eigenfactor. The range of the eleven 
(31%) journals that had an Eigenfactor was from 0.000399 to 0.021091. The mean score 
was 0.003710 and the median was 0.001550. 

All journals had an /i-index from Publish or Perish. The indices ranged from 1 to 29. The 
mean index was 9 and the median was 7. 
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The number of articles published from 2008-2011 ranged from 8 to more than 1,000. The 
Journal of Technology Education and the Journal of Undergraduate Neuroscience Education 
published 8. Bioscience and Science Education published the most articles (more than 
1,000). The mean number of articles published (substituting 1,000 articles for the journals 
that actually published more than 1,000) was 218; the median was 214. 


Table 4. Citation Data for Selected Agriculture, Natural Resources, and Life Sciences Journals 
Publishing SoTL. 


Journal Title 

2010 

1SI 

Impact 

Factor 

2011 

Eigen- 

factor 

Score 

2008- 
2011 
PoP ft- 
index 

2008- 

2011 

PoP # 
of 

articles 

Advances in Physiology Education 

1.382 

0.00155 

10 

225 

American Biology Teacher 

0.09 

0.00048 

7 

587 

Applied Engineering in Agriculture 

0.507 

0.00246 

7 

183 

Biochemistry & Molecular Biology Education 

0.619 

0.00066 

6 

387 

Bioscene: Journal of College Biology Teaching 

N/A 

N/A 

2 

21 

BioScience 

5.51 

0.02109 

29* 

1,000 

Bioscience Education 

N/A 

N/A 

4 

56 

CBE Life Sciences Education 

N/A 

N/A 

1 

26 

Electronic Journal of Science Education 

N/A 

N/A 

5 

53 

Environmental Education Research 

0.679 

N/A 

15 

197 

Instructional Science 

1.473 

0.00138 

14 

165 

International Journal of Science Education 

1.063 

0.00338 

19 

518 

Journal of Agricultural Education 

N/A 

N/A 

8 

432 

Journal of Agricultural Education & Extension 

N/A 

N/A 

7 

238 

Journal of Biological Education 

0.367 

0.0004 

6 

134 

Journal of College Science Teaching 

N/A 

N/A 

8 

168 

Journal of Environmental Education 

0.316 

N/A 

11 

253 

Journal of International Agricultural & Extension 
Education 

N/A 

N/A 

4 

25 

Journal of Microbiology & Biology Education 

N/A 

N/A 

6 

129 

Journal of Natural Resources & Life Sciences 

Education 

N/A 

N/A 

3 

116 

Journal of Research in Science Teaching 

2.728 

0.00424 

23 

264 

Journal of Science Education & Technology 

N/A 

N/A 

15 

298 

Journal of Science Teacher Education 

N/A 

N/A 

10 

214 

Journal of sTEm Teacher Education 

N/A 

N/A 

4 

19 

Journal of Technology Education 

N/A 

N/A 

3 

8 

Journal of Technology Studies 

N/A 

N/A 

2 

18 
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Journal of Undergraduate Neuroscience Education 

N/A 

N/A 

2 

8 

Journal of Women and Minorities in Science and 
Engineering 

N/A 

N/A 

4 

75 

NACTA (North American Colleges and Teachers of 
Agriculture) Journal 

N/A 

N/A 

2 

22 

Research in Science & Technological Education 

N/A 

N/A 

6 

95 

Research in Science Education 

0.853 

0.001 

14 

280 

School Science & Mathematics 

N/A 

N/A 

6 

176 

Science Education 

1.9 

0.00417 

26* 

1,000 

Science Educator 

N/A 

N/A 

3 

28 

Science Teacher Education 

N/A 

N/A 

10 

208 

Studies in Science Education 

1.267 

N/A 

8 

43 


* Google Scholar limit of 1,000 articles reached, so h-index may not be accurate. 


Discussion 


Acceptance Rate 

The acceptance rates for the journals varied greatly, from 7-80%. This could be an 
indicator of journal quality if the premise is that higher quality journals have lower 
acceptance rates. However, the editor of the Journal of Biological Education indicated that 
acceptance rates for his journal varied significantly from month to month based on the 
quality of submissions. This agreed with some other journal editors' comments. The editor 
of the Journal of International Agricultural and Extension Education reported acceptance 
rates of 18% (2007), 20% (2008), and 10% (2009), showing some variation from year to 
year. The policy of Environmental Education Research was not to publicize the acceptance 
rate because of concerns the editor stated that authors and reviewers could strictly, yet 
falsely, equate lower acceptance rate with higher quality. 

The acceptance rate does not factor in other realities of publishing, such as the possibility of 
receiving a high volume of high quality manuscripts worthy of publishing (possible higher 
acceptance rate), receiving generally low quality manuscripts (possible lower acceptance 
rate), or editorial philosophies that provide more mentoring for authors through the 
preparation of manuscripts. Editors must make decisions about accepting a variety of topics 
for issues, and may need to reject quality papers based on subject content decisions. 
Scientific journals that publish articles about SoTL might have higher acceptance rates than 
education journals since high consensus fields tend to have higher article acceptance rates 
(Hargens, 1988). 

Longevity 

When the authors began this study, they expected to find that many SoTL journals were new 
publications. However, the journals identified for this study started in every decade of the 
20 th century, with the newest initiating publication in 2003. This is because the scope of the 
journals is the broader field of education, not solely the scholarship of teaching and learning. 
It appears that the disciplines of agriculture, natural resources, and the life sciences initiated 
few new journals focusing on SoTL. 
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Open Access Availability 

Most of the journals (n = 26, 72%) were not available open access. Therefore, there may be 
fewer citations to articles in these journals, affecting citation analysis metrics. 

Inclusion in Indexing/abstracting Services 

The primary indexing/abstracting database for education, ERIC, included the most journals 
(n = 28, 76%). The science indexing/abstracting databases covered few (CAB, n = 6, 17%; 
Web of Science, n = 5, 14%; BIOSIS, n = 2, 6%). Although SoTL is an interdisciplinary area 
that blends education with other academic disciplines, the education indexing resources 
cover its literature much more comprehensively for the fields covered by this paper. This 
may be one reason why faculty who do not publish in the scholarship of teaching and 
learning may be unfamiliar with its primary journals. 

Five (14%) of the journals were not included in any of these indexing sources, however, it 
was not possible to discern whether the journals were indexed in Agricola due to database 
reconstruction at the time of the study. These journals may be included in that important 
resource. Another possible explanation is that the journals may consider selection for 
inclusion in these sources as less important than in the past due to the widespread use of 
Google Scholar for finding articles. 

Citation Analysis 

Most of the journals did not have an ISI journal impact factor (n = 22, 61%). This is due to 
the lack of inclusion of many education journals in Web of Science. JCR coverage of 
disciplines varies, and tends towards established disciplines (e.g., Agronomy, Biology, 
Forestry). Cross-disciplinary areas, especially those that span the sciences and the social 
sciences (i.e., SoTL in the life sciences) do not fit well into the criteria for inclusion of 
journals. 

The ISI journal impact factor ranged from .09-5.51. The mean was 1.34 and the median 
0.96. The ISI journal impact factor is the number of citations in a journal in 2010 divided by 
the number of articles published in 2008 and 2009 (ISI's Journal Citation Reports help 
page). This is an indication that authors tend not to cite the articles in these journals soon 
after publication. But considering the volume of articles published in the scholarly literature 
each year, the chance of being cited would be small. 

As a comparison, all education and educational research journals in JCR had a median 
impact factor of 0.649. Agriculture journals had a median impact factor of 0.410 and 
biology journals, 1.339. 

This reflects on the limitations of the ISI journal impact factor, which relies on traditional 
mono-disciplinary boundaries and long-established publishing timelines. Given the rapid 
changes in the availability of citations, and sometimes full text, through the internet, there 
is a need to develop new criteria beyond the narrow confines of ISI journal impact factor. 

Most of the journals did not have an Eigenfactor (n = 25, 69%). This is because they were 
not indexed by ISI. The score ranged from .000399-.021091. The mean was .003710 and 
the median .001550. This could be the expected value given the number of citations 
produced each year. But the sample of values was small and many values were missing 
(n = 25, 69%). This affected the interpretation of these results. 
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The journals published a widely ranging number of articles during the period from 2008- 
2011. A possible explanation is that the journals with the fewest articles (n = 8), Journal of 
Technology Education and the Journal of Undergraduate Neuroscience Education, covered 
more specific topics than the journals with the most articles, Bioscience and Science 
Education (n = more than 1,000). 


Conclusion 

This study examined commonly accepted criteria for evaluating SoTL journals in the areas of 
agriculture, natural resources, and the life sciences. There was great variation in the 
acceptance rates of journals. Most of the journals were at least 10 years old and most were 
not available through open access. ERIC was the primary indexing source that included the 
journals. Most journals did not have an ISI journal impact factor or Eigenfactor. The 
number of articles published in each journal varied greatly, from 8 to over 1,000. 

The large percentage of missing values for ISI journal impact factor and Eigenfactor™ Score 
affected the ability to interpret these results. 

There were several limitations to this study. There was no established "core list" of journals 
in SoTL for agriculture, natural resources, or the life sciences. The authors compiled the list 
used for this study by comparing existing lists of journals on SoTL web sites of land-grant 
universities that were members of the Committee on Institutional Cooperation. However, 
these lists did not provide any indication of how the compilers selected the journals. It is 
possible that they were compiled from expert opinion or from existing lists of other 
institutions. The latter exemplifies "institutional isomorphism," in which institutions tend to 
imitate others those to whom they aspire due to economic and professional pressure (Dey, 
Milem, & Berger, 1997). 

The study did not take into consideration the number of hits to or downloads of articles. 

This is a difficult metric to obtain, particularly for subscription journals. However, it would 
provide a dimension that is particularly relevant in the online environment. 

It was not possible to obtain a list of journals indexed in the Agricola database because the 
National Agricultural Library was in the process of reengineering the database. Since 
Agricola is one of the primary resources for articles on agriculture, this left a gap in data for 
the indexing criterion. 

Future studies might examine the following topics: 

- A bibliometric study of the journals that publish the most on SoTL in agriculture, 
natural resources and the life sciences. If Bradford's law of scattering applies, then a 
small number of journals will publish the majority of the articles about the 

topic.(Bradford, 1985). 

- SoTL is fundamentally about improving the quality of teaching. What role does 
improved teaching through research play in the evaluation process in agriculture, 
natural resources, and life sciences disciplines? 

- How do the findings from this study about agriculture, natural resources, and life 
sciences SoTL journals compare with SoTL journals in other disciplines? 
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■ Do journals about science that publish articles about SoTL have higher acceptance 
rates than journals about education research that publish articles about science? 

■ What other metrics can reflect the quality of journals in which authors in the 
scholarship of SoTL in agriculture, natural resources, and the life sciences? 

* How can the inherent intra-disciplinary differences in a field such as SoTL in 

agriculture, natural resources, and the life sciences be reconciled when promotion 
and tenure evaluations take place? 


In conclusion, this paper highlighted the criteria that are available for evaluating journal 
quality, or at least understanding the nature of a journal in context to similar titles. These 
criteria may be useful in providing a rationale for the selection of journals in which to submit 
articles for publication. 


References 

ABS launches academic journal quality guide version 4. Retrieved from 
http ://www.the-abs.org.uk/?id=257 

Altmann, K., & Gorman, G. (1998). The usefulness of impact factors in serial selection: A 
rank and mean analysis using ecology journals. Library Acquisitions: Practice & 
Theory, 22, 147-159. 

Becker, W. E., & Andrews, M. L. (2004). Introduction The scholarship of teaching and 
learning in higher education: Contributions of research universities (pp. 1-8). 
Bloomington, IN: Indiana University Press. 

BIOSIS previews - subject categories. Retrieved from 

http://science.thomsonreuters.com/cgi-bin/jrnlst/jlsubcatg.cgi?PC=BP 

Blake, V. L. P. (1996). The perceived prestige of professional journals, 1995: A replication of 
the Kohl-Davis study. Education for Information, 14, 157-179. 

Blessinger, K., & Frasier, M. (2007). Analysis of a decade in library literature: 1994-2004. 
College & Research Libraries, 68, 155-169. 

Blessinger, K., & Hrycaj, P. (2010). Highly cited articles in library and information science: 

An analysis of content and authorship trends. Library & Information Science 
Research, 32, 156-162. 

Bowden, R. G. (2007). Scholarship reconsidered: Reconsidered. Journal of the Scholarship 
of Teaching and Learning, 7, 1-21. 

Boyer, E. L. (1990). Scholarship reconsidered: Carnegie Foundation for the Advancement of 
Teaching Princeton, NJ. 


https://doi.org/l0.20429/ijsotl.20l2.060209 


IJ-SoTL, Vol. 6 [ 2012 ], No. 2 , Art. 9 


Bradford, S. C. (1985). Sources of information on specific subjects. Journal of Information 
Science, 10, 173-180. 

Braxton, J. M., Luckey, W., & Helland, P. (2002). Institutionalizing a broader view of 

scholarship through boyer's four domains. San Francisco, CA: Jossey-Bass Inc. Pub. 

Bray, N. J., & Major, C. H. (2011). Status of journals in the field of higher education. The 
Journal of Higher Education, 82, 479-503. 

CAB Abstracts serials cited. Retrieved from 

http://www.ca bi.org/defa ult.aspx?site=170&page= 1016&pid = 125 

Centra, J. A. (1977). How universities evaluate faculty performance: A survey of 
department heads. GRE Board Research Report GREB. 

Davis, P. M., Lewenstein, B. V., Simon, D. H., Booth, J. G., & Connolly, M. J. L. (2008). 
Open access publishing, article downloads, and citations: Randomised controlled 
trial. British Medical Journal, 337, 343-345. 

deVries, S., Kelly, R., & Storm, P. M. (2010). Moving beyond citation analysis: How surveys 
and interviews enhance, enrich, and expand your research findings. College & 
Research Libraries, 71, 456-466. 

Dey, E. L., Milem, J. F., & Berger, J. B. (1997). Changing patterns of publication 

productivity: Accumulative advantage or institutional isomorphism? Sociology of 
Education, 308-323. 

Edgerton, R. (2005). Foreward. In K. A. O'Meara & R. E. Rice (Eds.), Faculty priorities 

reconsidered: Rewarding multiple forms of scholarship (pp. xi-xiv). San Francisco: 
Jossey-Bass 

Eliot, C. W. (1901). Educational reform: Essays and addresses: The Century co. 

Eysenbach, G. (2006). Citation advantage of open access articles. Plos Biology, 4, el57. 
doi: 10.1371/journal.pbio.0040157 

Fairweather, J. S. (1993) Teaching, research, and faculty rewards: A summary of the 

research findings of the faculty profile project. University Park, PA: National Center 
on Postsecondary Teaching, Learning and Assessment, Penn State University. 

Finnegan, D. E., & Gamson, Z. F. (1996). Disciplinary adaptations to research culture in 
comprehensive institutions. In D. E. Finnegan, D. Webster & Z. F. Gamson (Eds.), 
Faculty and faculty issues in colleges and universities (2nd ed., pp. 476-498). 
Boston, MA: Pearson Custom Publishing. 

Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA: The 
Journal of the American Medical Association, 295, 90-93. doi: 

10.1001/jama.295.1.90 


https://doi.org/l0.20429/ijsotl.2012.060209 


19 


Criteria for Evaluating Journals in the Scholarship of Teaching and Learning 


Glassick, C. E., Huber, M. T., Maeroff, G. I., & Teaching, C. F. f. t. A. o. (1997). Scholarship 
assessed : Evaluation of the professoriate (1st ed.). San Francisco: Jossey-Bass. 

Goodyear, R. K., Brewer, D. J., Gallagher, K. S., Tracey, T. J. G., Claiborn, C. D., 

Lichtenberg, J. W., & Wampold, B. E. (2009). The intellectual foundations of 
education. Educational Researcher, 38, 700-706. doi: 10.3102/0013189X09354778 

Greider, A. P. (2002). An introduction to the literature and general sources. In B. S. 

Hutchinson & A. Greider (Eds.), Using the agricultural, environmental, and food 
literature (pp. x, 533 p.). New York: Marcel Dekker. 

Haensly, P. J., Hodges, P. E., & Davenport, S. A. (2009). Acceptance rates and journal 
quality: An analysis of journals in economics and finance. Journal of Business & 
Finance Librarianship, 14, 2-31. 

Hagstrom, W. (2001). Science Encyclopedia of sociology (2nd ed., Vol. 4, pp. 2455-2463). 
New York: Macmillan Reference USA. 

Hargens, L. L. (1988). Scholarly consensus and journal rejection rates. American 
Sociological Review, 53, 139-151. 

Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. 

Proceedings of the National Academy of Sciences of the United States of America, 

102, 16569. 

Hutchings, P., Huber, M. T., & Ciccone, A. (2011). The scholarship of teaching and learning 
reconsidered: Institutional integration and impact. San Francisco: Jossey-Bass. 

ISI's Journal Citation Reports help page. Retrieved from http://admin- 
apps.webofknowledge.com.login.ezproxy.lib.purdue.edu/JCR/JCR 

Journals indexed in ERIC. Retrieved from 

http://www.eric.ed.gov/ERICWebPortal/journalList/journalList.jsp 

Kohl, D. F., & Davis, C. H. (1985). Ratings of journals by arl library directors and deans of 
library and information science schools. College and Research Libraries, 46, 40-47. 

Ladwig, J. P., & Sommese, A. J. (2005). Using cited half-life to adjust download statistics. 
College & Research Libraries, 66, 527-542. 

Lamp, J. W. (2009). Journal ranking and the dreams of academics. Online Information 
Review, 33, 827-830. 

Leydesdorff, L. (2008). Caveats for the use of citation indicators in research and journal 
evaluations. Journal of the American Society for Information Science and 
Technology, 59, 278-287. 

Long, S. S. (2010). The scholarly journal: Will peer reivew still function and how? Journal of 
Scholarly Publishing, 42, 67-70. 


https://doi.org/10.20429/ijsotl.2012.060209 


20 


IJ-SoTL, Vol. 6 [ 2012 ], No. 2 , Art. 9 


Master journal list. Retrieved from http://ip-science.thomsonreuters.com/mjl/ 

McGarty, C. (2000). The citation impact factor in social psychology: A bad statistic that 
encourages bad science. Current Research in Social Psychology, 5, 1-16. 

Morrill Act of 1890. The Agricultural College Act of 1890. An act to apply a portion of the 
proceeds of the public lands to the more complete endowment and support of the 
colleges for the benefit of agriculture and mechanic arts... United States Statues at 
Large, Pub. L. No. 51-249, 30 August 1890 417-419 (1890). 

Morrill Act. An act denating public lands to the several states and territories which may 
provide colleges for the benefit of agriculture and the mechanic arts, 12, United 
States Statutues at Large, Pub. L. No. 37-130 July 1862 503-505 (1862). 

Nisonger, T. E., & Davis, C. H. (2005). The perception of library and information science 
journals by Ms education deans and arl library directors: A replication of the Kohl- 
Davis study. College & Research Libraries, 66, 341-377. 

Norris, M., Oppenheim, C., & Rowland, F. (2008). The citation advantage of open-access 

articles. Journal of the American Society for Information Science and Technology, 59, 
1963-1972. 

O'Meara, K. (2005). Effectis of encouraging multiple forms of scholarship nationwide and 
across institutional types. In K. O'Meara & R. E. Rice (Eds.), Faculty priorities 
reconsidered: Rewarding multiple forms of scholarship (pp. 255-289). San Francisco, 
CA: Jossey-Bass. 

Paynter, R. A., Jackson, R. M., & Mullen, L. B. (2010). Core journal lists: Classic tool, new 
relevance. Behavioral & Social Sciences Librarian, 29, 15-31. doi: 
10.1080/01639260903571096 

Potter, J. (2008). Starting with the discipline. In R. Murray (Ed.), The scholarship of 

teaching and learning in higher education. Berkshire, England: University Press. 

Publish or Perish. Retrieved from http://www.harzing.eom/pop.htm#download 

Serenko, A., & Dohan, M. (2011). Comparing the expert survey and citation impact journal 
ranking methods: Example from the field of artificial intelligence. Journal of 
Informetrics, 5, 629-648. 

Shulman, L. S. (2000). From minsk to pinsk: Why a scholarship of teaching and learning? 
Journal of Scholarship of Teaching and Learning, 1, 48-53. 

Shulman, L. S. (2004). Visions of the possible: Models for campus support of the 

scholarship of teaching and learning. In W. E. Becker & M. L. Andrews (Eds.), The 
scholarship of teaching and learning in higher education: Contributions of research 
universities (pp. 9-23). Bloomington, IN: Indiana University Press. 

Smith, K., & Middleton, M. (2009). Australian library & information studies (Ms) researchers 
ranking of Ms journals. Australian Academic & Research Libraries, 40, 1-21. 


https://doi.org/l0.20429/ijsotl.2012.060209 


21 


Criteria for Evaluating Journals in the Scholarship of Teaching and Learning 


Social Sciences Citation Index. Retrieved from 

http://thomsonreuters.com/products_services/science/science_products/a- 

z/social_sciences_citation_index 

Stankus, T., Clavin, J. M., & Joslin, R. (1999). The best original scientific research, review, 
methods and symposia journals with their current web addresses ranked within their 
primary subject category. Science & Technology Libraries, 18, 111-183. 

Swan, A. (2010). The open access citation advantage: Studies and results to date. 

Retrieved from 

http://eprints.Ecs.Soton.Ac.Uk/18516/2/citation_advantage_paper.Pdf. 

Theall, M. (2003). College teaching. In J. W. Guthrie (Ed.), Encyclopedia of education (2nd 
edition ed., pp. 413-417.). New York: Macmillan Reference USA. 

Toutkoushian, R. K. (2001). Changes in the demand for public land-grant universities: The 
use of panel datasets. Research in Higher Education, 42, 679-707. 

Towns, M. H., & Kraft, A. (2012). The 2010 rankings of chemical education and science 
education journals by faculty engaged in chemical education research. Journal of 
Chemical Education. 

Ugaz, A. G., Boyd, C. T., Croft, V. F., Carrigan, E. E., & Anderson, K. M. (2010). Basic list of 
veterinary medical serials: Using a decision matrix to update the core list of 
veterinary journals. Journal of the Medical Library Association, 98, 282-292. 

Van Nierop, E. (2009). Why do statistics journals have low impact factors? Statistica 
Neerlandica, 63, 52-62. 

Web of Science help. Retrieved from 

http://images.webofknowledge.com/WOKRS541B2/help/WOS/hp_citation_report_hin 

dex.html 

Weiner, S. A. (2009). Tale of two databases: The history of federally-funded information 
systems for education and medicine. Government Information Quarterly, 26, 450- 
458. doi: 10.1016/giq.2009.02.003 

Weissinger, T. (2010). The core journal concept in black studies. The Journal of Academic 
Librarianship, 36, 119-124. 

Youngen, G. K. (2011). Multidisciplinary journal usage in veterinary medicine: Identifying 
the complementary core. Science & Technology Libraries, 30, 194-201. doi: 

10.1080/0194262X.2011.569257 


Appendix 

List of Journals on the Scholarship of Teaching and Learning in Agricultural, Natural 
Resources, and the Life Science Included in Analysis with ISSN's. 


https://doi.org/10.20429/ijsotl.2012.060209 


22 


IJ-SoTL, Vol. 6 [ 2012 ], No. 2 , Art. 9 


Title 

ISSN 
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(online 

journal) 

Advances in Physiology Education 

1043-4046 

1522-1229 

American Biology Teacher 

0002-7685 

1938-4211 

Applied Engineering in Agriculture 

0883-8542 

none cited 

Biochemistry & Molecular Biology Education 

1470-8175 

1539-3429 

Bioscene: Journal of College Biology Teaching 

1539-2422 

none cited 

BioScience 

0006-3568 

0006-3568 

Bioscience Education 

no print 

1479-7860 

CBE Life Sciences Education 

no print 

1931-7913 

Electronic Journal of Science Education 

no print 

1087-3430 
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1469-5871 
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1042-0541 
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1098-2736 
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1573-1839 

Journal of Science Teacher Education 

1046-560X 

1573-1847 

Journal of sTEm Teacher Education 
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2158-6594 

Journal of Technology Education 

1045-1064 

none cited 

Journal of Technology Studies 

no print 

1071-6084 

Journal of Undergraduate Neuroscience Education 

no print 

1544-2896 
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1094-3277 

none cited 
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